Foundational High-level Static Analysis

A formal method (e.g., of software verification) is foundational if it proves program properties from the axioms of logic and from a low-level machine specification (ISA or transistors). The proofs should be machine-checked, because hand-checked proofs don’t track real software systems well. With recent advances on several fronts (in static analysis, semantics, compiler verification) it is now feasible to put scalable, fully automatic program analyses (such as shape analysis of concurrent C programs) on a foundational footing. This is an exciting time for the formal verification of software, in part because several threads of research, in progress for decades, have the potential to cohere. These threads include a gradual revolution in the specification methods for operational semantics of programming languages (1994– 2008); progress in the specification of weak memory models (1992–2008); steady progress in abstract interpretation (1978-2008); the maturation of mechanized proof assistants (1978-2008) and dependently typed logics (1988-2008); successes in compiler verification (1989, 2006); and finally, enough decades of Moore’s law so that the proof assistants and the abstract interpretations finally have usable performance. Now we can achieve end-to-end guarantees: based on fully automated static analyses of source programs, we can efficiently obtain machine-checked proofs about the behavior of machine language. In this paper I will outline the architecture of one such end-to-end system for concurrent programs. The system itself is not at present connected end-toend; here I outline the possible, not report an achieved result. However, each of the components has been built, by different researchers at different institutions in different countries, and the method of connecting them has become clear.1 A top-to-bottom verified architecture. At the top we have a C program that uses malloc/free, pointers, threads and locks, all with some conventional discipline. At the bottom we have the machine language of an instruction set architecture 1 This is not a survey! At each level I will cite one or two illustrative examples, but of course there are many more that I don’t have space to cite here. (ISA). (In fact, we can go higher than C and lower than the ISA, as I’ll discuss at the end.) We apply a modern automatic static analysis algorithm to the C program. This can guarantee important safety properties with little or moderate effort from the programmer: error messages from static-analyzer can usually give appropriate feedback in the programming process. For example, “lock l is always held whenever shared variable x is accessed.” More sophisticated analyses can track dynamic patterns of lock-to-data correspondence, and can work even in the presence of pointers and aliasing. As a case study, we will choose a particular static analysis. A shape analysis is an analysis of how a program uses pointer data structures. For example one of the classic papers on this topic explains in its title, “Is it a tree, a DAG, or a cyclic graph?” [5] More recent shape analyses include those of Gotsman et al. [7] or that of Guo et al. [8]. Both of these algorithms appeared in PLDI’07 and represent the state of the art. Gotsman’s algorithm proves that a concurrent program accesses data only when it holds the relevant lock; Guo’s algorithm deduces the backpointer/crosspointer/downpointer invariants of complex linked data structures, for use in a parallelizing compiler. Even more recent is Yang et al. [12] which scales up a separation-logic-based shape analysis algorithm so that it can verify real programs up to 10,000 lines of code.2 They write, “It identifies memory safety errors and memory leaks in several Windows and Linux drivers and, after these bugs are fixed, it automatically proves integrity of pointer manipulation for these drivers.” “Proves integrity?” Many static analyzers are unsound but useful—they catch some erroneous programs, but permit some buggy programs to slip through—we can characterize them as “bug-finding tools.” In contrast, we are interesed in provably sound analyses, as are Gotsman, Guo, and Yang, whose analyses all come with soundness proofs: if the analyzer finds invariants about a program, then those invariants must hold in the operational semantics of the source language. However, these soundness proofs are problematic for several reasons: 2 Years ago, such algorithms were impractical to imagine because they would require hundreds of megabytes of memory. In contrast, Yang’s algorithm is practical today because it runs in only a gigabyte!